Risk management is a process that allows any associate within or outside of a technology and operations domain to balance the operational and economic costs of protective measures while protecting the operations environment that supports the mission of an organization. Risk is the net negative impact of the exercise of vulnerability, considering both the probability and the impact of occurrence.
An organization typically has a mission. Risk management plays an important role in protecting against an organization's operational risk losses or failures. An effective risk management process is an important component of any operational program. The principal goal of an organization's risk management process should be to protect against operational losses and failures, and ultimately the organization and its ability to perform the mission.
Within the financial industry, the Basel II Capital Accord requires firms to capture key Business Environment and Internal Control Factors (BEICF) that can change its operational risk profile. These factors will make an organization's risk environments, help align capital assessments with risk management objectives, and recognize both improvements and deterioration in operational risk profiles in a more immediate fashion. To qualify for regulatory capital purposes the use of these factors in an organization's risk measurement framework must meet standards. Over time, the process and outcomes need to be validated through comparison to actual internal loss experience and appropriate adjustments made.
Under the United States final rule (issued in November 2007 by inter-agencies including FRB (Federal Reserve Board) and OCC (Office of the Comptroller of the Currency) to implement risk-based capital requirements in the United States for large, internationally active banking organizations), an organization has flexibility in the approach it uses to conduct its BEICFs. As such, the methods for conducting comparisons of these assessments against actual operational loss experience may also vary and precise modeling calibration may not be practical. It may still be important for an organization to perform such comparisons to ensure that its assessments are current, reasonable, and appropriately factored into the organization's AMA framework. In addition, the comparisons could highlight the need for potential adjustments to the organization's operational risk management processes.
Back-testing is the comparison of forecasts to realized outcomes. Any risk assessment system is considered well calibrated if the (ex-ante) estimated risk assessment measures deviate only marginally from what has been observed ex-post. The challenge is how to quantify the deviation and how to perform the comparison given all the subtle wrinkles presented by operational risks.
Currently, based on industry benchmarking and from regulators, most organizations and banks may perform either a qualitative review or a simple trend analysis (comparing trends of risks against trends of losses to derive subjective opinions and qualitative outputs). No organization currently back-tests subjective operational risk and control assessments against objective losses quantitatively and use the model output to (a) adjust risk-based capital, (b) forecast step-ahead losses from risk and control assessments and (c) validate the accuracy of the assessments.